Multi-label Feature Selection Research Articles

Multi-label feature selection via spectral clustering-based label enhancement and manifold distribution consistency

Multi-label feature selection via spectral clustering-based label enhancement and manifold distribution consistency

Feature selection for multilabel classification with missing labels via multi-scale fusion fuzzy uncertainty measures

Numerous high-dimension multilabel data are generated, posing a challenge for multilabel learning. Building effective learning models with discriminative features is essential to improve the performance of multilabel learning. Multilabel feature selection can filter out the discriminative features according to their contribution to classification. However, ambiguity, uncertainty, and missing labels coexist in real-life multilabel data, which brings adverse effects to multilabel feature selection. The multi-scale fuzzy rough set gives an effective way to mine intrinsic knowledge hidden in uncertain data. This paper first extends the multi-scale learning to multilabel data with missing labels and proposes a feature selection method for multilabel classification with missing labels via multi-scale fusion fuzzy uncertainty measures called FSMML. The missing label space construction and feature evaluation metric are carefully investigated in the framework of multi-scale learning. A multilabel multi-scale learning strategy is formalized with the fuzzy granularity cognitive mechanism as the core, and the multi-scale fusion fuzzy label learning is given to reconstruct the missing label space. Then, a novel multilabel multi-scale fuzzy rough sets with missing labels is developed, and the significance of each scale is quantified. Moreover, some multi-scale fusion fuzzy uncertainty measures are defined by capturing the sample fuzzy similarity in the feature and reconstructed label spaces. Accordingly, the relevance between features and label set and the interactivity and redundancy between features in feature evaluation are discussed. Finally, FSMML chooses high-quality features to maximize relevance and interactivity and minimize redundancy. Extensive experiments demonstrate the effectiveness of FSMML on fifteen datasets with missing labels.

Partial multi-label feature selection via low-rank and sparse factorization with manifold learning

Feature selection is a commonly utilized methodology in multi-label learning (MLL) for tackling the challenge of high-dimensional data. Accurate annotation of relevant labels is crucial for successful multi-label feature selection (MFS). Nevertheless, multi-label datasets frequently consist of ground-truth and noisy labels in real-world applications, giving rise to the partial multi-label learning (PML) problem. The inclusion of noisy labels complicates the task of conventional MFS methods in accurately identifying the optimal features subset in such datasets. To tackle this issue, we propose a novel partial multi-label feature selection method with low-rank sparse factorization and manifold learning, called PMFS-LRS. Specifically, we first decompose the candidate label matrix into two distinct components: a low-rank matrix referring to ground-truth labels and a sparse matrix referring to noisy labels. This decomposition allows PMFS-LRS to effectively distinguish noise labels from ground-truth labels, thereby mitigating the impact of noisy data. Then, the local label correlations are explored using a manifold learning framework to improve the label disambiguation performance. Finally, a l2,1-norm regularization is integrated into the objective function to facilitate effective feature selection. Comprehensive experiments conducted on both real-world and synthetic PML datasets demonstrate that PMFS-LRS is superior to several existing state-of-the-art MFS methods.

Discriminative label correlation based robust structure learning for multi-label feature selection

Feature selection is a key technique to tackle the curse of dimensionality in multi-label learning. Lots of embedded multi-label feature selection methods have been developed. However, they face challenges in identifying and excluding redundant features. To address these issues, this paper proposes a multi-label feature selection method that combines robust structural learning and discriminative label regularization. The proposed method starts from the feature space rather than data space, motivated by the principle that redundant features have high similarity or strong correlation. To exclude redundant features, a regularization on the feature selection matrix is designed by combining ℓ2,1-norm penalty with inner products of feature weight vectors. This regularization can help to learn a robust structure in the feature selection matrix. Meanwhile, both of the similarity and dissimilarity of labels of instances are involved in exploring discriminative label correlations. Extensive experiments verified the effectiveness of the proposed model for feature selection.

Sparse semi-supervised multi-label feature selection based on latent representation

Sparse semi-supervised multi-label feature selection based on latent representation

Label relaxation and shared information for multi-label feature selection

Due to the rapid growth of labels and high-dimensional data, multi-label feature selection has attracted increasing attention. However, two common issues are ignored by existing multi-label feature selection methods: 1) The pseudo-label matrix is constructed by using logical label matrix directly. 2) There is an imbalance of label density in multi-label data. To tackle the mentioned issues, we innovatively propose a new method named Label relaxation and Shared information for Multi-label Feature Selection (LSMFS). Specifically, LSMFS combines a logical label matrix with a non-negative label relaxation matrix to fit a pseudo-label matrix, which is used for learning the correlations of class labels. LSMFS uses the feature weight matrix to capture a shared information of different related labels, which mitigates the influence of low-density labels on feature selection. The above principles are transformed into the objective function of LSMFS, and an alternate iterative optimization algorithm is developed to solve the objective function. Experiments on multi-label datasets from different domains have demonstrated the effectiveness of the proposed method LSMFS. Code is released at https://github.com/HQUF/LSMFS.

Fast Multilabel Feature Selection via Global Relevance and Redundancy Optimization.

Information theoretical-based methods have attracted a great attention in recent years and gained promising results for multilabel feature selection (MLFS). Nevertheless, most of the existing methods consider a heuristic way to the grid search of important features, and they may also suffer from the issue of fully utilizing labeling information. Thus, they are probable to deliver a suboptimal result with heavy computational burden. In this article, we propose a general optimization framework global relevance and redundancy optimization (GRRO) to solve the learning problem. The main technical contribution in GRRO is a formulation for MLFS while feature relevance, label relevance (i.e., label correlation), and feature redundancy are taken into account, which can avoid repetitive entropy calculations to obtain a global optimal solution efficiently. To further improve the efficiency, we extend GRRO to filter out inessential labels and features, thus facilitating fast MLFS. We call the extension as GRROfast, in which the key insights are twofold: 1) promising labels and related relevant features are investigated to reduce ineffective calculations in terms of features, even labels and 2) the framework of GRRO is reconstructed to generate the optimal result with an ensemble. Moreover, our proposed algorithms have an excellent mechanism for exploiting the inherent properties of multilabel data; specifically, we provide a formulation to enhance the proposal with label-specific features. Extensive experiments clearly reveal the effectiveness and efficiency of our proposed algorithms.

Feature selection for multi-label learning based on variable-degree multi-granulation decision-theoretic rough sets

Multi-label learning (MLL) suffers from the high-dimensional feature space teeming with irrelevant and redundant features. To tackle this, several multi-label feature selection (MLFS) algorithms have emerged as vital preprocessing steps. Nonetheless, existing MLFS methods have their shortcomings. Primarily, while they excel at harnessing label-feature relationships, they often struggle to leverage inter-feature information effectively. Secondly, numerous MLFS approaches overlook the uncertainty in the boundary domain, despite its critical role in identifying high-quality features. To address these issues, this paper introduces a novel MLFS algorithm, named VMFS. It innovatively integrates multi-granulation rough sets with three-way decision, leveraging multi-granularity decision-theoretic rough sets (MGDRS) with variable degrees for optimal performance. Initially, we construct coarse decision (RDC), fine decision (RDF), and uncertainty decision (RDU) functions for each object based on MGDRS with variable degrees. These decision functions then quantify the dependence of attribute subsets, considering both deterministic and uncertain aspects. Finally, we employ the dependency to assess attribute importance and rank them accordingly. Our proposed method has undergone rigorous evaluation on various standard multi-label datasets, demonstrating its superiority. Experimental results consistently show that VMFS significantly outperforms other algorithms on most datasets, underscoring its effectiveness and reliability in multi-label learning tasks.

Multi-label Feature Selection based on Label-specific features and Manifold Learning

Each instance in multi-label data is associated with multiple labels, and there are irrelevant or redundant features in its feature space, which leads to the performance degradation of multi-label learning algorithms. Multi-label feature selection selects representative features from the feature space to improve the accuracy of the model. Due to the high cost of labels and the difficulty of data collection, there will be some missing labels in the data set, which affects the accuracy of feature selection. To solve this problem, a multi-label feature selection algorithm based on label-specific features and manifold learning is proposed. The algorithm uses the linear relationship between the features and labels in known label samples to build a linear regression model for learning label-specific features. By using the nonlinear relation between instances and the nonlinear relation between features, we can precisely learn the label-specific features. We use the Laplacian feature mapping method to construct the instance manifold model and the feature manifold model, which are also used as the regular term constraint weight matrix. The final model can not only complete the missing labels, but also select sparse and representative features. The feature selection is carried out by analyzing the weight of the feature given by the final model. Experiments were conducted to verify the effectiveness of the proposed algorithm under different label deletion rates on four evaluation indexes.

Double-Layer Hybrid-Label Identification Feature Selection for Multi-View Multi-Label Learning

Multi-view multi-label feature selection aims to select informative features where the data are collected from multiple sources with multiple interdependent class labels. For fully exploiting multi-view information, most prior works mainly focus on the common part in the ideal circumstance. However, the inconsistent part hidden in each view, including noises and specific elements, may affect the quality of mapping between labels and feature representations. Meanwhile, ignoring the specific part might lead to a suboptimal result, as each label is supposed to possess specific characteristics of its own. To deal with the double problems in multi-view multi-label feature selection, we propose a unified loss function which is a totally splitting structure for observed labels as hybrid labels that is, common labels, view-to-all specific labels and noisy labels, and the view-to-all specific labels further splits into several specific labels of each view. The proposed method simultaneously considers the consistency and complementarity of different views. Through exploring the feature weights of hybrid labels, the mapping relationships between labels and features can be established sequentially based on their attributes. Additionally, the interrelatedness among hybrid labels is also investigated and injected into the function. Specific to the specific labels of each view, we construct the novel regularization paradigm incorporating logic operations. Finally, the convergence of the result is proved after applying the multiplicative update rules. Experiments on six datasets demonstrate the effectiveness and superiority of our method compared with the state-of-the-art methods.

LSFSR: Local label correlation-based sparse multilabel feature selection with feature redundancy

In recent studies, existing multilabel feature selection models have focused on either considering the relationship between labels or the redundancy between features. Furthermore, they only use simple sparsity constraints to process high-dimensional data without the intrinsic relationships between features and labels. These issues can have a great impact on the classification effectiveness of feature selection. To address these limitations, this article describes a new local label correlation-based sparse multilabel feature selection approach with feature redundancy. First, a new loss function is established among the matrices of samples, label coefficients, and labels. Then, the Frobenius norm is imposed to investigate the potential relationships between features and labels. The weight matrix is sparsified by the l2,1 norm to ensure that the new loss function has high interpretability. Second, a manifold constraint is employed to capture the local geometric structure between labels and to delve deeper into the latent information among the local labels. Then manifold constraints and Laplacian scores are combined for embedding feature selection to guide the exploration of hidden latent label. Finally, by considering the differences between the feature scores and the redundancy between the samples, feature redundancy is analyzed via the modified cosine similarity, and a candidate feature subset with low redundancy is generated. The l2 norm is used to select features with low redundancy while preserving sparsity, and a novel objective function is developed to optimize this solution. Thus, a sparse feature selection algorithm via local label correlation and feature redundancy is designed, and has demonstrated remarkable classification effectiveness in comparative experiments conducted on 21 multilabel datasets.

Dynamic multi-label feature selection algorithm based on label importance and label correlation

AbstractMulti-label distribution is a popular direction in current machine learning research and is relevant to many practical problems. In multi-label learning, samples are usually described by high-dimensional features, many of which are redundant or invalid. This paper proposes a multi-label static feature selection algorithm to solve the problems caused by high-dimensional features of multi-label learning samples. This algorithm is based on label importance and label relevance, and improves the neighborhood rough set model. One reason for using neighborhood rough sets is that feature selection using neighborhood rough sets does not require any prior knowledge of the feature space structure. Another reason is that it does not destroy the neighborhood and order structure of the data when processing multi-label data. The method of mutual information is used to achieve the extension from single labels to multiple labels in the multi-label neighborhood; through this method, the label importance and label relevance of multi-label data are connected. In addition, in the multi-label task scenario, features may be interdependent and interrelated, and features often arrive incrementally or can be extracted continuously; we call these flow features. Traditional static feature selection algorithms do not handle flow features well. Therefore, this paper proposes a dynamic feature selection algorithm for flow features, which is based on previous static feature selection algorithms. The proposed static and dynamic algorithms have been tested on a multi-label learning task set and the experimental results show the effectiveness of both algorithms.

Multi-label feature selection via latent representation learning and dynamic graph constraints

As an effective method to deal with the curse of dimensionality, multi-label feature selection aims to select the most representative subset of features by eliminating unfavorable features. Although great progress has been made in this field, how to mine adequate supervisory information from multi-label data remains a key challenge. Compared to the latent information of instances, the latent information of instance relevance contains both the basic information of instances and the latent relevance between instances. Base on this knowledge, we propose a novel multi-label feature selection method named LRDG that explores latent representation learning and dynamic graph constraints. Specifically, we introduce the latent representation of instance relevance as supervisory information for pseudo-label learning, and minimize information loss during pseudo-label learning by means of the label manifold, the non-negative constraints, and the minimization of the Frobenius norm between pseudo-labels and ground-truth labels. In addition, considering the shortcomings brought by traditional graph regularization, we propose to use the dynamic graph constructed from low-dimensional pseudo-labels to constrain feature weights. Extensive experiments on various multi-label datasets demonstrate the effectiveness of the proposed method. The code is available at https://github.com/yunbao520/LRDG.

Semi-supervised multi-label feature selection based on dual dynamic graph

Traditional multi-label feature selection is usually performed under the condition of given label information, but nowadays, labeling multi-label data is a huge project, which is both time-consuming and labor-intensive, but if there is no label information condition, it will lead to poor feature selection, in order to solve this problem, this paper proposes a new semi-supervised multi-label feature selection method, i.e., semi-supervised multi-label feature selection algorithm based on dual dynamic graph. In this paper, a semi-supervised multi-label feature selection algorithm is proposed by constructing a dual dynamic graph. First, the method selects the most discriminative features for dimensionality reduction through the feature selection method of least squares regression, combined with the redundancy penalty of highly correlated features. Second, the label information is added to the construction of sample matrix similarity to learn the similarity. A semi-supervised multi-label feature selection framework is constructed by designing iterative updates of dual dynamic graphs to learn more accurate pseudo-label matrices to guide feature selection. Finally, the paper validates the above model using the alternating iteration optimization algorithm and verifies the effectiveness of the algorithm through experiments.

Multi-label feature selection based on fuzzy rough sets with metric learning and label enhancement

Multi-label feature selection based on fuzzy rough sets, as a key step of multi-label data preprocessing, has been widely concerned by scholars in recent years. Most of the existing multi-label feature selection algorithms directly treat labels as logical labels and use a single distance metric to describe similarity. However, the variability of label descriptions and the limitations of a single distance measure should not be overlooked. In this paper, we propose a fuzzy rough set model with metric learning and label enhancement. Specifically, we use a kernel membership label enhancement algorithm based on JS divergence to convert logical labels into numerical labels, which not only reflects the importance of different labels, but also takes into account the differences in label distribution. In addition, a multi-metric learning algorithm is proposed for multi-label learning, in which the metric distance function under the label space and feature space can be learned autonomously. Then, based on the proposed model, we propose a novel multi-label feature selection algorithm based on metric learning and fuzzy rough sets. On this basis, a fast multi-label feature selection algorithm is further designed to improve the computational efficiency. In the experiments, compared with other nine algorithms on real-world datasets, the results show the superiority of the proposed algorithm.

Multi-Label Feature Selection via Positive or Negative Correlation

Multi-Label Feature Selection via Positive or Negative Correlation

Fuzzy information gain ratio-based multi-label feature selection with label correlation

Fuzzy information gain ratio-based multi-label feature selection with label correlation

Multi-label feature selection with global and local label correlation

In various application domains, high-dimensional multi-label data has become more prevalent, presenting two significant challenges: instances with high-dimensional features and a large number of labels. In the context of multi-label feature selection, the objective is to choose a subset of features from a given set that is highly pertinent for predicting multiple labels or categories associated with each instance. However, certain characteristics of multi-label classification, such as label dependencies and imbalanced label distribution, have often been overlooked although they hold valuable insights for designing effective multi-label feature selection algorithms. In this paper, we propose a feature selection model which exploits explicit global and local label correlations to select discriminative features across multiple labels. In addition, by representing the feature matrix and label matrix in a shared latent space, the model aims to capture the underlying correlations between features and labels. The shared representation can reveal common patterns or relationships that exist across multiple labels and features. An objective function involving L2,1-norm regularization is formulated, and an alternating optimization-based iterative algorithm is designed to obtain the sparse coefficients for multi-label feature selection. The proposed method was evaluated on 14 real-world multi-label datasets using six evaluation metrics, through comprehensive experiments. The results indicate its effectiveness, surpassing that of several representative methods.

Multi-label feature selection with high-sparse personalized and low-redundancy shared common features

Prevalent multi-label feature selection (MLFS) approaches to obtain the most suitable feature subset by dealing with two issues, namely sparsity and redundancy. In this paper, we design an efficient Elastic net based high Sparse personalized and low Redundancy Feature Selection approach for multi-label data named ESRFS to address the two obstacles, i.e., low-sparse LASSO-norm leads to personalized features for each label while high redundancy l2,1-norm explore shared common features for all labels in multi-label learning. These two problems impede the selection of high-quality features for classification. In comparison with previous MLFS approaches, ESRFS has two main superiority. First, ESRFS achieves higher sparse personalized features than LASSO-norm. Second, ESRFS can identify low redundancy shared common features with strong discrimination by introducing a novel regularization term. To effectively and efficiently identify the most optimal feature subset, an alternating-multiplier-based rule is introduced to optimize ESRFS. Experimental results on fifteen multi-label data sets show that ESRFS can achieve obvious superior performance compared to eight state-of-the-art MLFS approaches in 80%, 80%, 73.3%, 80%, 86.7%, 80% cases based on Hamming Loss, Zero-One Loss using MLkNN, Micro-F1 and Macro-F1 using SVM as well as Micro-F1 and Macro-F1 using 3NN perspectives.

Multi-label Feature selection with adaptive graph learning and label information enhancement

The high dimensionality and complexity of multi-label data make obtaining accurate label sets in practical applications difficult. Noisy data in the labels will affect the model’s classification performance. Existing methods only reconstruct logical labels into semantic labels to mitigate the effect of inherent noise and do not specifically address the noise. In order to solve this problem and eliminate the effect of noise, this paper proposes a multi-label feature selection method with adaptive graph learning and label information enhancement (AGLE). First, each label is represented as a combination of other labels using global label correlation, which can reconstruct the original label matrix to obtain the self-expression label matrix. Based on this, similar labels in two matrices can be considered reliable labels, and not similar may be noise. Therefore, constructing a predictive label matrix to weigh the learning of original and self-expression labels can enhance reliable information. Second, an adaptive graph is utilized to learn global label correlation to ensure that the self-expression label matrix can be continuously optimized. Then, the local label manifold structure is utilized to constrain the adaptive graph for more stable performance. Finally, the adaptive graph and ℓ2,1-norm are combined to guide learning the feature weight matrix. For the proposed method, an iterative algorithm based on alternating optimization is used to compute the weight matrix. Comparing experimental results with several state-of-the-art algorithms demonstrates the effectiveness of the AGLE method for the noise problem.